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ABSTRACT 

The philosophy and purpose of ground simulation tests for un- 
manned spacecraft, as used at the Goddard Space Flight Center, is re- 
viewed. Laboratory test results are presented from 16 prototype and 
48 flight spacecraft. The summarized results show a four-to-one ratio 
in problems per spacecraft for prototype compared to flight models, 
and for both models the simulated space test has revealed the largest 
number of problems. A comparison of the number of space problems 
with test problems on the same spacecraft shows no correlation and 
shows that 100% trouble-free operation was not obtained on any space- 
craft. Data from simulated space testing of 270 experiments for an 
observatory program show an exponential relationship of malfunctions 
with time. The curve can be used to estimate the number of problems 
that will be detected by varying the test time, but cannot be extrapolated 
to long-term (days) testing. The data from the systems test of a com- 
plete observatory under simulated space conditions show failures oc- 
curring after 12 days of testing and verify the need for long-term 
systems tests. 
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EFFECTIVENESS OF ENVIRONMENT-SIMULATION 
TESTING FOR SPACECRAFT* 

by 

John C. New and A. R. Timmins 
Goddard Space Flight Center 


INTRODUCTION 

Prior to the space age, test and evaluation usually involved large sample sizes from which 
probabilities could be assigned to the chances for success of an item, a weapon, or weapon system. 
Working with spacecraft, however, presents a much different problem. Here the sample size 
amounts to two or possibly three items. Another major difference is that the item itself is tested, 
any difficulties or malfunctions remedied, and the tested item itself flown. Conventional statistics 
are not amenable to demonstrating a stated reliability. Under these conditions, it was necessary 
to develop a new approach to testing and a new test philosophy. This paper reviews the test phi- 
losophy used at Goddard Space Flight Center (GSFC) and presents data which summarizes the en- 
tire test and evaluation experience of the Center. The latter includes experience from industry- 
conducted programs. 


TEST AND EVALUATION PHILOSOPHY 

It would be ideal if the space environment could be reproduced exactly for spacecraft evalua- 
tion. However, economic, technological, and terrestrial limitations prevent achievement of this 
ideal objective. The following examples show the kinds of limitations indicated. To simulate ex- 
actly the cold of space requires a facility capable of providing a temperature of 4°K over an area 
of 200 square feet (and up). Careful analysis of the effect of compromising this temperature shows 
that a practical, economic answer is possible. By using liquid nitrogen (boiling point, 77°K) as the 
cryogenic fluid, the space thermal radiation environment can be duplicated with an error of less 
than one percent. Some of the technological problems, however, do not have such a desirable solu- 
tion. In the case of the micrometeoroid and energetic particles space environments, a lack of 
facilities requires that these environments be dealt with from an experience and theoretical stand- 
point. The same is true for the terrestrial limitations where gravity presents a formidable prob- 
lem in gaining test information on weightlessness. It is then necessary to devise a philosophy of 
testing which accommodates these restraints and achieves a high degree of assurance that the 
spacecraft will be successful. 


* Presented at the American Institute of Aeronautics and Astronautics and the American Astronautical Society Stepping Stones to Mars 
Meeting, Baltimore, Maryland, March 1966. 
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At GSFC, the test philosophy is to concentrate on total systems testing under environmentally 
induced stress levels for sufficient time to detect most deficiencies. To qualify a given design, a 
prototype system is tested at augmented stress levels and durations. The actual flight system is 
tested at environmental stress levels which would not be expected to be exceeded more than once in 
20 cases. For design qualification tests on prototypes, it is customary to increase vibration ampli- 
tudes by 50 percent and durations twice that expected during launch. Predicted temperature ex- 
tremes are extended 10°C to provide a safety margin. Vacuum simulation is of the order of 10“ 5 
to 10 " 8 torr. 


—INFANT MORTALITY- 
DESIGN 

WORKMANSHIP ] 
MATERIAL 


INTEGRATION 


FAILURES 


| E NVIRONMENTAL 
TESTING 


-RANDOM FAILURES — H 


— LAUNCH 


SPACE FLIGHT - 



RANDOM 

FAILURE RATE / | 

V' 





SPACECRAFT LIFE 


Figure 1— Failure pattern. 


Since duplication of the planned orbital life 
(usually one year) is not economically feasible, 
the duration of the simulated orbital tests is 
limited to that time (five to twelve days) neces- 
sary to detect and correct failures attributable 
to M infant mortality.” Such tests include tem- 
perature extremes, solar simulation, spacecraft 
positioning for various orbital conditions, and 
operation of the complete system in all modes 
of transmission. The test philosophy hypothe- 
sizes a failure pattern with time (as shown in 
Figure 1). The operating time accumulated on 
a flight unit before launch ranges between 600 
and 1200 hours. Complete details on the test 
philosophy are provided in Reference 1. 


TEST AND EVALUATION PROGRAM 


Adequate facilities are required to conduct an effective test and evaluation program. Means 
must be available for determining the effect of shock, vibration, temperature, humidity, and the 
simulated space environment as well as determining the balance, moment of inertia, and other 
physical characteristics of the spacecraft. These are the usual environments and characteristics 
that need to be investigated, and the facilities required for investigation have become somewhat 
standardized. On the other hand, as spacecraft become larger, and as improved environment sim- 
ulation becomes necessary, the need for advanced or specialized facilities develops. Three 
facilities of this type are either in use or under construction at GSFC. One of these is called the 
Space Environment Simulator (SES). It is a nominal 50,000 cubic foot thermal-vacuum chamber 
with major working dimensions of 27.5 feet in diameter by 40 feet in height. It has a vacuum ca- 
pability of 1 X 10 ~ 8 torr, liquid nitrogen and gaseous helium cryogenic walls to simulate the cold 
of space, and a modular solar simulation (mercury-xenon) capability up to 1.5 solar constants. 
Reference 2 provides further details, and Figure 2 shows one view of the SES. This facility is in 
use and two tests, one of 44 days duration, have been completed. 


Figure 3 shows a facility for magnetic measurements and effects. It is a forty-foot, three- 
axis Braunbek coil system which can accommodate a complete spacecraft. It has a uniform field 
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Figure 2— Space Environment- Simulator. 


Figure 3— Spacecraft Magnetic Test Facility. 


to 0.5 gamma over a six-foot sphere, and five gamma over a 10-foot sphere. It can simulate from 
zero to twice the earth T s magnetic field for simulated space measurements, cancel the earth's 
magnetic field, check the magnetic moment of the spacecraft, and test active magnetic attitude 
control systems. 

The significance of combined environments has been discussed for some time and is somewhat 
controversial. The launch environment is a prime example of combined environments, and the sim- 
ulation of this environment is expected to be both scientifically and practically rewarding. A spe- 
cialized facility to provide this combined environment, called the Launch Phase Simulator, is shown 
in Figure 4. It will handle a complete spacecraft 
weighing up to 5000 pounds, and will be capable of 

exposing the complete spacecraft to the combined ji 

environments of launch acceleration, mechanical vS| 

motion (three -axis), acoustic excitation, and glfc a , . 

launch pressure profile in nearly real time. 


Test and evaluation is only one part of the 
complete development cycle of a spacecraft pro- 
gram (shown in Figure 5). In this hypothetical 
case, the time period for test and evaluation is 
of the order of 12 months. Figure 6 shows actual 




Figure 4— Launch Phase Simulator (Reference 3). 
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Figure 5— Scientific spacecraft development cycle. 
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Figure 7— Comparison of spacecraft mission requirements. 
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Figure 6 — Period for environmental test program. 


time for the test and evaluation of 7 spacecraft 
programs conducted at GSFC. The illustration 
shows considerable variation in the time for dif- 
ferent programs. One special case was accom- 
plished in 30 days. In others, over 18 months 
may be required. The problem, of course, is to 
achieve adequate assurance in minimum time. 

To develop an adequate test and evaluation 
program, consistent with overall program re- 
quirements, it is necessary to classify spacecraft 
performance requirements. For instance, the 
amount, duration, and extent of testing devoted 
to a sounding rocket is of a different magnitude 
than a scientific spacecraft, and the latter is of a 
different magnitude than a manned spacecraft. 
Figure 7 compares various requirements for 
different kinds of spacecraft. With this per- 
spective, the effectiveness of a comprehensive 
system test policy is reviewed. The test and 
evaluation results include scientific, meteor- 
ology, and communications spacecraft. 
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EFFECTIVENESS OF ENVIRONMENT SIMULATION TESTS 

A good examination of test and evaluation philosophy is provided by dealing with many samples. 
Such an examination is provided by reviewing the test results of all spacecraft under the direction 
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of GSFC launched in the period of 1960-1964. For this review, the number of problems have been 
used as a correlative index. A problem is defined as any item which causes a delay or rework 
during design qualification and acceptance testing of the spacecraft. 


The validity of GSFC test philosophy is attested by the results from the evaluation of 16 pro- 
totype and 48 flight units. A total of 855 problems were uncovered and corrected, resulting in 27 
successful spacecraft or probes and only one major failure - Syncom - in a four-year period. Ten 


of these spacecraft were designed, developed, and 
tested at GSFC, and 18 of them were under GSFC 
management, but the actual design, development, 
and tests were conducted by prime contractors. 
The space record, shown in Figure 8, is updated 
beyond the time period covered by the laboratory g 
test results by including satellites launched S 
through December 1965. ^ 

LL_ 

o 

The summarized information from 64 space- g 

CO 

craft tested at GSFC, or by industry under the § 
management of GSFC, is shown in Figure 9. The z 
summary shows a ratio of approximately 4 to 1 
for the number of problems per spacecraft for 
the prototype model as compared to the flight 
model. The value of having a prototype space- 
craft is amply demonstrated by the large reduc- 
tion in number of problems per flight spacecraft 
compared to prototype spacecraft. Although 
neither the ratio nor its parts are firm values, 
they can be used as a basis for future compari- 
sons, and possibly as an indication of when the 
philosophy needs reexamination. 

Figure 10 contains the same summarized in- 
formation subdivided to show a comparison of the 
results for spacecraft tested at GSFC compared 
to spacecraft tested by industry. The illustration 
shows a ratio of approximately 5 to 1 for the 
GSFC prototype to flight model spacecraft, and a 
ratio of approximately 3 to 1 for industry. No 
particular significance is attached to the differ- 
ence in ratios at the present time. The value of 
the ratios is to provide a basis for comparing 
past, present, and future programs. Figure 10 
also shows the distribution of problems on each 
spacecraft by environment. Most of the problems 
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Figure 8— GSFC spacecraft record (43 of 47 launches 
placed spacecraft in orbit; 40 of 43 spacecraft in or- 
bit were successful). 
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Figure 9— Spacecraft test problems (prototype 
vs. flight model). 
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Figure 10— Spacecraft test problems (by environment). 
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have been detected in the simulated space en- 
vironment, and a desirable learning curve is 
demonstrated by the decrease in the number of 
problems in this environment from the prototype 
to the flight model spacecraft. The use of a 
structural model spacecraft for testing prior to 
the prototype undoubtedly helps to reduce the 
number of problems encountered with the pro- 
totype in the vibration environment. 

Although the number of problems per flight 
spacecraft looks small by comparison, the re- 
sults of the first ten spacecraft launched by 
GSFC are used to determine whether or not all 
the problems were detected and corrected. A 
comparison of the number of problems en- 
countered on the flight units in the laboratory 
compared to the number of problems on the 
same flight units in space is shown in Figure 11. 
The number of problems encountered in the 
laboratory varies from 1 to 28 for various pro- 
grams. In space, the maximum number of 
problems encountered is 6. Note that a problem 
in space is not necessarily identical with a fail- 
ure in space. A problem is defined here as any 
performance in space outside of design param- 
eters. This overly severe definition is used to 
obtain the maximum information available from 
GSFC space flight experience. Actually, sci- 
entific information was obtained from each of 
the 10 spacecraft shown in Figure 11. Seven of 
the 10 operated for the full planned lifetime of 
the satellite, and the other three had lifetimes 
of 112, 193, and 312 days. 


Figure 11— Comparison of space problems with 

test problems for ten spacecraft. An appraisal of this space flight experience 

shows several areas in which the information 

gained can be used to achieve even more successful performance: (a) About 25 percent of the 

space problems were not subjected to relevant systems tests. For example, in one case the nose 
cone outgassed and changed the thermal coating properties of the spacecraft. This caused over- 
heating and failure of an experiment, (b) In some cases, a relevant systems test was not possible. 
For instance, no simulation capability is available for testing stability problems caused by solar 
radiation pressure or aerodynamic forces at perigee altitudes, (c) Flight devices which cannot be 
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operated during systems tests require special attention. Explosively actuated devices are an ex- 
ample. On one launch an antenna failed to deploy; post-launch investigation revealed a marginal 
firing current for the explosively actuated device used for deployment. 

The space experience analysis is indirect. For many problems the evidence is quite conclu- 
sive, for others, post-flight tests have been helpful in assessment, and for others, the reason is 
obscure or unknown. The preceding paragraph lists some positive areas in which improved per- 
formance can be achieved. However, there are other space failures or problems not revealed by 
the comparatively short-term simulated space tests in the laboratory. These are the problems, 
apparently time dependent, which occur after a long time (10 to 300 days) in space, but for which 
no failure analysis is available. The beginning of the wear-out period for space hardware has not 
been established, but it is significant that the Alouette I satellite has operated satisfactorily for 
over three years in space, and is still operating. 


COST EFFECTIVENESS 

A typical development cycle for scientific spacecraft is shown in Figure 6. Test and evalua- 
tion occupies a nominal time period of 12 months. This duration, of course, depends upon many 
factors and, as shown in Figure 7, varies from 1 to 19 months for 7 spacecraft programs. Figure 
12 shows actual distribution of the time required to complete 33 programs. The illustration shows 
that test and evaluation is completed in less than 6 months for the majority of the programs. On 
the other hand, there are programs that require as much as two years for test and evaluation. 

With increased emphasis on lower cost, the time period allotted to test and evaluation is being 
given additional scrutiny. It is of interest to conjecture what effectiveness could be achieved if the 
long-term programs could be shortened to the norm, and also what savings could be accrued if the 
norm could be reduced significantly. Considerable savings can be accomplished by shortening the 
test program, but reliability cannot be sacrificed just for the sake of saving time. Data show that 
simulated space testing at the systems level accounts for most of the problems during this phase 
of a system test program. To determine whether the number of these problems could be reduced 
by additional emphasis on subsystem testing, or 
whether the problems are of a different nature, 
subsystem testing of an observatory type pro- 
gram was examined. In the program, 270 ex- 
periments were subjected to a simulated space 
test at subsystem level. Figure 13 shows the 
distribution of problems encountered as a func- 
tion of time for the simulated space test. Per- 
formance of one of the observatories from this 
program under a simulated testing environment 
was also examined. The number of problems with 
respect to time under the simulated environment 
is shown in Figure 14. A total of 17 problems 



Figure 12— Time for systems testing of spacecraft. 
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Figure 1 3— Experiment malfunction rate under 
simulated space environment. 

lems discovered in a flight model of a system, i 
the failures (and the problems) and to have som< 



Figure 14— System malfunctions versus time under 
simulated space environment. 


were detected, eight of which were classified as 
failures. Ten of the 17 problems were with ex- 
periments. Of the ten, six had no problems in 
subsystem test, and the four others did. How- 
ever, in only one case was the problem possibly 
associated with the same problem in subsystem 
testing. The data are not sufficient to make any 
firm conclusions on the duration and amount of 
subsystem testing required, but are presented 
to illustrate the type of information needed for 
such conclusions. Another aspect of cost effec- 
tiveness can be inferred from the data presented 
That is, in order to reduce the number of prob- 
will be necessary to analyze the specific cause of 
mechanism to enable the use of this information 


in other programs at the time when the problem can be circumvented. 


CONCLUSIONS 

The needs of the space program with its emphasis on limited production of highly complex and 
highly reliable systems demand a new philosophy of test and evaluation. A philosophy has been 
developed which places primary emphasis on complete systems tests under realistic environment 
simulation. Confidence in the design is attained through tests of working prototype systems at aug- 
mented stress levels up to 50 percent; confidence in flight readiness is achieved for flight units by 
tests under the expected environments of launch or space. The laboratory and space results on 
unmanned scientific spacecraft over a four-year period show the philosophy to be sound. The cost 
effectiveness of this philosophy is still being evaluated. The philosophy is dependent upon hav- 
ing test articles that can be repaired; a fairly large capital investment in simulation facilities, 


8 



and a resourceful and competent staff which can respond quickly after problems are 
detected. 


Goddard Space Flight Center 
National Aeronautics and Space Administration 
Greenbelt, Maryland, August 31, 1966 
124-12-03-01-51 
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conducted so as to contribute ... to the expansion of human knowl- 
edge of phenomena in the atmosphere and space. The Administration 
shall provide for the widest practicable and appropriate dissemination 
of information concerning its activities and the results thereof ” 

— National Aeronautics and Space Act of 1958 
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